AI Admin Portal

Projects

Learnings

Scenarios

Editor

New Scenario

Basic Information
Short scenario name shown to learners and admins.
Role of the person the learner is speaking with (e.g. coworker, patient, manager).
Display name of the agent, the character the learner talks to (e.g. Jordan). Published to the Agent_Name Storyline variable. Optional; the module shows "Agent" when blank.
Briefing the learner sees before starting. Written in second person ("You are...").
Behaviors the learner is evaluated on. Visible to the learner; the rubric maps to these.
One-shot Configuration
Prompts shown to the learner. Their response to these is what gets graded.
Optional. Items the learner ranks in priority order before submitting.
Grading Criteria (Admin Only)
Detailed grading criteria for the post-hoc grading agent. Admin-only; never seen by the learner or the role-play AI.
Optional grading constraints. Admin-only.
Factual scenario context for the grading agent. Third-person and omniscient. Not addressed to the learner.

Test Scenario

Your Response

Grading your response...

Delete Scenario?

This action cannot be undone.

New Learning

Leave empty to auto-generate. Cannot be changed after creation.

Delete Learning?

This action cannot be undone.

Feedback Evaluation Results

Running feedback evaluations... This may take 30-60 seconds.

Generating test responses...

Test Results

Avg Score
-
Pass Rate
-

Details

Select a test result to view details

Conversational Feedback Evaluation

Saved Transcripts

Count
-
Pass Rate
-
Avg Score
-

Details

Select a saved transcript to view details

Paste Configuration

Paste the scenario configuration JavaScript code from the old Playground:

Sample Responses Editor

Edit the 10 sample responses that will be used for feedback evaluation. These responses will be reused across multiple test runs.

AI Prompt Preview

This is the exact prompt that was sent to OpenAI for grading:

Project Settings

Changes here may take up to 5 minutes to take effect on new conversations.

Conversational Scenarios

End Detection Prompt

Default

Runs after every learner turn to decide if the conversation has ended. Applies to every conversational scenario in this project. Per-scenario qualifiers can be added in the scenario editor.

End Detection Model

Default

OpenAI model used by the end-detection classifier. Latency-sensitive — runs on every turn. Use a fast classifier-grade model. See OpenAI models.

Grading Model

Default

OpenAI model used to grade conversational transcripts after the session ends. Not latency-sensitive — pick the smartest model.

Oneshot Scenarios

Grading Model

Default

Storyline Setup Info

Everything needed to configure a Storyline module for this project. Create each variable listed below in Storyline's variable panel, then paste the bootstrap loader into a JavaScript trigger.