Scenario Demo

Author: raghav-2002-os

README.md

@@ -76,7 +76,7 @@ There are three main ways to use PromptWizard:
 - Scenario 2 : Generating synthetic examples and using them to optimize prompts
 - Scenario 3 : Optimizing prompts with training data
 
-**NOTE** : Refer this [notebook](demos/scenarios/dataset_scenarios_demo.ipynb) to get a detailed understanding of the usage for each of the scenarios. **This servers as a starting point to understand the usage of PromptWizard**
+**NOTE** : Refer this [notebook](demos/scenarios/dataset_scenarios_demo.ipynb) to get a detailed understanding of the usage for each of the scenarios. **This serves as a starting point to understand the usage of PromptWizard**
 
 #### High level overview of using PromptWizard
 - Decide your scenario

demos/scenarios/dataset_scenarios_demo.ipynb

@@ -125,7 +125,7 @@
     "config_dict = {\n",
     "                \"task_description\": \"You are a mathematics expert. You will be given a mathematics problem which you need to solve\",\n",
     "                \"base_instruction\": \"Lets think step by step.\",\n",
-    "                \"mutation_rounds\": 2\n",
+    "                \"mutation_rounds\": 5\n",
     "               }\n",
     "update_yaml_file(file_path,config_dict)"
    ]
@@ -274,7 +274,8 @@
    "id": "96d07ae3",
    "metadata": {},
    "source": [
-    "Set the configurations to first generate synthetic training data"
+    "Set the configurations to first generate synthetic training data. \\\n",
+    "Any number of synthetic examples can be generated and then used for optimizing prompts as mentioned in STEP 2"
    ]
   },
   {
@@ -287,7 +288,7 @@
     "file_path = 'configs/promptopt_config.yaml' \n",
     "# Set the number of synthetic training examples to be generated\n",
     "config_dict = {\n",
-    "                \"num_train_examples\":40\n",
+    "                \"num_train_examples\":20\n",
     "               }\n",
     "update_yaml_file(file_path,config_dict)"
    ]

promptwizard/glue/promptopt/techniques/critique_n_refine/core_logic.py

@@ -469,14 +469,14 @@ def get_best_prompt(self, params: PromptOptimizationParams,use_examples=False,ru
                                 f"current_base_instruction: {current_base_instruction}")
                 candidate_prompts = self.gen_different_styles(current_base_instruction,
                                                             params.task_description,
-                                                            params.mutation_rounds,
+                                                            params.mutation_rounds+1,
                                                             params.style_variation)
 
                 if run_without_train_examples:
                     prompt_index = 1
                     print("\nOptimization Finished...")
                     print("\nPossible prompt variations:")
-                    for candidate in candidate_prompts[:5]:
+                    for candidate in candidate_prompts[:params.mutation_rounds]:
                         final_best_prompt = self.prompt_pool.final_prompt.format(
                         instruction=candidate,
                         answer_format=params.answer_format,