CHANGELOG.md

Changelog

0.5.0 (2025-05-06)

Features

• add a check for whether values in score_df are NaN (#756) (d9cc780)
• add DocDev for auto-generating workspace documentation (#781) (bcba6ea)
• add drafting pipeline (#832) (efedddf)
• add mlflow logger in RD loop to log (#815) (b91b54f)
• add naive experiment generator and update proposal configurations (#759) (75494f4)
• add reasoning_effort parameter to LiteLLMAPIBackend and LLMSett… (#754) (113889f)
• add reviewer in feedback (#765) (1a95bee)
• archive python and csv files in workspace to maintain results (#814) (67d0e01)
• checkpoint selection (#744) (a15a06a)
• dump model (#776) (b49481e)
• idea pool integrated to exp_gen & add timer to RD-Agent & pause-resume to RD-loops (#795) (e62aefa)
• joblib cache (#749) (83a0411)
• merge failed and successful traces together (#766) (3a2aa8c)
• propose hypothesis across multiple parts in pipeline (#827) (acb0e21)
• pull image with progress (#777) (5cad086)
• raise error when timeout in api call (#793) (eafd4df)
• reanalyze competition info & pipeline coding evaluator prompt (#837) (f7b5258)
• refine merge (#842) (99463b4)
• refine prompt (#760) (a91b182)
• trace merging (#836) (a3d5473)
• using different chat model in different part (#822) (c052ea6)

Bug Fixes

• adapting UI to mock trace (#841) (8a5754c)
• add time to timer when api timeout bug (#826) (f45d6ae)
• add wait_retry to exp_gen v2 (#783) (b9fb7cf)
• align competion_full_desc and scenario_all_desc, remove redundant info in problems proposal (#808) (76d8536)
• bug fix in timer start (#807) (9af7161)
• bug in problem identification (#806) (e1d5a29)
• duplicate model names test in pipeline coder & runner (#763) (be3ee9d)
• fix a bug in docker result extraction (#824) (e1c0f98)
• fix competition metric direction (#784) (3be0057)
• fix model input shape bug and costeer_model bug (#821) (b34bd89)
• fix some minor bugs (#758) (963f96e)
• fix some minor bugs in qlib scenario (#817) (79962a7)
• import path of T (#787) (ac008a6)
• improve eval alignment check (e.g. small-scale finetuning) (#802) (d391578)
• Set PYTHONPATH in env.run_ret_code call in FBWorkspace class (#755) (68b5018)
• task_gen for better understanding (#752) (6bfc1e5)
• update feedback.py (#772) (133778c)
• update metric direction to return bool (#791) (0bf365e)
• update runner max loop to 1 in DS scenario (#820) (3da378e)

0.4.0 (2025-04-04)

Features

• (Kaggle) add base template for competition: tabular-playground-series-may-2022 (#481) (f3405ca)
• a unified CoSTEER to fit more scenarios (#491) (cddbd02)
• add a new competition (#474) (2fc0d77)
• add a tool to enable saving workspace files into a specific folder (#728) (bca864b)
• add baseline score stat (#590) (2948026)
• add configurable volume mode for Docker volumes in env.py (#537) (642a022)
• add constraint labels for semantic search (#680) (0584cfc)
• add cross validation to workflow (#700) (82e9b00)
• add describe_data_folder_v2 (#738) (bc8e846)
• add do_truncate control for the load function (#656) (2b960a5)
• add do_truncate control for the load function (#656) (2b960a5)
• add eda to data science scenario (#639) (35aa479)
• add hypothesis guidelines and rule-based ranking (#746) (c077b82)
• Add line length limit to shrink_text function and settings (#715) (75ed5e1)
• add loop_n parameter to the main loop (#611) (778c166)
• add max time config to costeer in data science (#645) (534686c)
• add mlebench submission validitor (#545) (712d94a)
• add model removal and adjust some framework logic (#681) (1edf881)
• add output_path to load function of LoopBase (#628) (dd33726)
• add pipeline coder (#742) (759f295)
• add rank into report (mle_summary) (#665) (13f7922)
• add restart and fix unzip (#538) (ed2c7d1)
• add retry mechanism with wait_retry decorator and refactor diff generation (#572) (de1cd72)
• add the shape of the CSV to the dataset description (#561) (a10c881)
• add timeout settings and cleanup step in data science runner (#539) (295abd5)
• add type checker to api backend & align litellm and old backend (#647) (d38eae9)
• align mlebench data and evaluation & several fix on kaggle workflow (#477) (f6c522b)
• backend: integrate LiteLLM API Backend (#564) (f477687)
• base data science scenario UI (#525) (39917b3)
• condaenv & full docker env (#668) (084dd6d)
• diff mode fix (#569) (0c509f5)
• display LLM prompt (#676) (8c93bba)
• Dynamically find and use sample submission file in eval tests (#542) (5f12b44)
• end-to-end optimization (#473) (d41343a)
• Enhance eval script with file cleanup and detailed submission checks (#529) (cf2ff92)
• exclude invalid session log folder (#554) (fa86e4d)
• improve the framework's ability to adaptively adjust the model (#629) (93806f3)
• independent use_azure_token_provider on chat and embedding (#452) (d223004)
• integrate azure deepseek r1 (#591) (e79ce5c)
• kaggle refactor (#489) (1b057d0)
• kaggle: several update in kaggle scenarios (#476) (245d211)
• loader prompt & simplify YAML loading and update data loader specifications (#736) (86f8bbf)
• make spec optional (#719) (a16b70f)
• Make system prompt role customizable in LLM settings (#632) (e4acd92)
• multi log folder, replace "epxx" in workspace path (#555) (8a69c9c)
• new exp gen v2 implementation (#725) (5dcc2d5)
• new-york-city-taxi-fare-prediction_template (#488) (a9caab7)
• out spec change for o1-preview (#666) (22894bd)
• refactor for general data science (#498) (7002dc4)
• refine logic for qlib_factor_from_report (#463) (21348d8)
• run benchmark on gpt-4o & llama 3.1 (#497) (64af0b5)
• summary and UI update (#581) (efa51f9)
• template changes for some kaggle competitions (#484) (2e38000)
• track and log accumulated completion cost in LiteLLMAPIBackend (#727) (b294a95)
• update prompts and descriptions for data science components (#731) (c20e226)
• variable printing tool of data_science coder testing (#658) (116c061)

Bug Fixes

• a default conf in scen qlib (#503) (d64a228)
• a small bug in exp_gen (#606) (f734dde)
• add check when retrying gen model codes (#699) (3b82f15)
• add DSExperiment type check and directory validation in log proc… (#535) (f59b12c)
• add ensemble test, change to "use cross-validation if possible" in workflow spec (#634) (acc97a8)
• add force parameter for cache_with_pickle & using cache when get kaggle leaderboard (#687) (c8841e5)
• add metric name check for valid scores (#724) (acc2ffb)
• add retry mechanism for GPU device check in DockerEnv (#573) (a780cfb)
• add scores.csv checking in ensemble_test (#567) (01808b4)
• add stdout context length setting and improve text shrinking logic (#559) (4ac26a6)
• align components' name (#701) (295a114)
• auto continue small bug (#598) (75eaecf)
• avoid try-except in ensemble eval prompts (#637) (5c58d6e)
• avoid warning for missing llama installation when not in use (#509) (5ec3422)
• change devault to default (#688) (7f401cd)
• change ensemble test (#622) (5de3595)
• change summary info of log folder (#552) (0eb258d)
• clarify an ambiguous explanation (#705) (5dbfc68)
• clarify cross_validation (#644) (906993e)
• coder prompt & model test text (#583) (0a41227)
• correct the configuration inheritance relationship (#671) (30b1ff8)
• default emb model (#702) (4329a72)
• direct_exp_gen to json_target_type in DSExpGen class (#661) (428b74a)
• docker error will trigger retry and data science runner loop set to 3 (#602) (ad785e0)
• ensure expected type (#593) (098a9a6)
• filter empty log traces in ds UI (#533) (1a2057c)
• fix a bug in cross validation (#618) (05a4f10)
• fix a bug in ensemble test script (#713) (ad32100)
• fix a bug in initial tasks (#635) (edb552e)
• fix a bug in kaggle conf (#459) (b4ed32b)
• fix a bug in progress_bar filter (#712) (ba5a84d)
• fix a bug in proposal (add last loop's exception to last task desc) (#596) (419186f)
• fix a bug in regular expression exception processing (#734) (67d3702)
• fix a bug in threshold score display (#592) (0b0a2dc)
• fix a bug related to model_name in ensemble (#692) (c6ce473)
• fix a minor bug (#694) (1405d8d)
• fix an error in model_coder prompt (#690) (4528826)
• fix combined_factors_df.pkl not loading in docker (#697) (3984b99)
• fix docs build error (#711) (c9e1d32)
• fix ExtendedSettingsConfigDict does not work (#660) (3a877f3)
• fix kaggle templates path error (#747) (3b3f504)
• fix KeyError direct_exp_gen (#735) (7200682)
• fix some bugs (ensemble output, HPO, model tuning) (#648) (818ee29)
• fix some bugs in the ensemble component (#595) (c0990ab)
• fix some bugs in workflow unit test (#624) (f845dcc)
• fix some description errors in direct_exp_gen (#698) (dfaacb6)
• fix some minor bugs and add AutoML & cross-validation (#604) (18c5ef2)
• fix submission file search and add TODO in env.py (#544) (54d930e)
• fix task return dict with wrong format (#558) (2008244)
• fix the errors in the coder and evaluator of the five components (#576) (c487f83)
• handle division by zero in percentage calculations (#550) (de16c91)
• handle invalid regex patterns in filter_progress_bar function (#579) (b0daee0)
• Handle ValueError when resolving relative path for uri (#585) (4c7765a)
• include data information in cache key generation (#566) (26dda46)
• keep some txt files (#557) (54aba85)
• mle_score save problem (#674) (ca2e478)
• move docker timeout message to __run() (#620) (585f4f9)
• move mlebench check into runner (#556) (b0f7965)
• move next_component_required logic to DSTrace class and accurate implement (#612) (c20d311)
• patching weird azure deployment (#494) (89c50ae)
• qlib and other scenario bugs (#636) (98de31d)
• refine prompt to generate the most simple task in init stage (#546) (9d6feed)
• replace API call with build_cls_from_json_with_retry function (#548) (eb72a47)
• replace func "len()" in ensemble test code to support various data type (#739) (ab9c7b9)
• return 1D embedding if create_embedding receive a string input (#670) (4a9c318)
• rich.print error when some control char in output (#684) (ec0cb2a)
• Runnable on first complete & Rename method to next_incomplete_component for clarity (#615) (93d9f63)
• runner COSTEER evaluator (#693) (6a379ec)
• save only one mle_score pkl for a running exp (#675) (f87ab67)
• Set default value for 'entry' parameter in Env.run method (#643) (e50d242)
• sort file name for cache reproduction (#588) (7158410)
• sota comparison logic (#608) (3575372)
• target json type & round (#662) (58cb58f)
• templates bug (#456) (434a868)
• trace summary df showing in dsapp (#551) (177096d)
• unzip kaggle data (#464) (3a9fc8e)

0.3.0 (2024-10-21)

Features

• add a new template for kaggle (#289) (eee3ab5)
• add download submission.csv button for kaggle scenario (#317) (dcdcbe4)
• add kaggle command (#271) (0938394)
• add kaggle tpl: feedback-prize (#331) (a288e39)
• add more templates for kaggle (#291) (da752ec)
• add normal rag into framework (#360) (91b0b1f)
• add qlib_factor_strategy (#307) (f8f59ff)
• Add ranking in kaggle scenario (#401) (b16b4be)
• Add runtime measurement for each step and loop in RDLoop. (#281) (83058c8)
• add s3e11 kaggle template (#324) (8c57524)
• Added RepoAnalyzer to empower auto-summary of a workspace (#264) (0bd349a)
• Added support for loading and storing RAG in Kaggle scenarios. (#269) (c4895de)
• announce Discord and WeChat (#367) (acac507)
• auto submit result after one kaggle RDLoop (#345) (ab55d70)
• better feedback & evaluation (#346) (cc9a8c1)
• Dynamic scenario based on task (#392) (665a037)
• Factor Implement Search Enhancement (#294) (4ecf25f)
• Feature selection v3 to support all actions (#280) (0047641)
• fix some bugs and add original features' description (#259) (1a5f45a)
• get kaggle notebooks & disscussion text for RAG (#371) (cead345)
• Iceberge competition (#372) (c10ea4f)
• implement isolated model feature selection loop (#370) (cf1292d)
• Initial version if Graph RAG in KAGGLE scenario (#301) (fd3c0fd)
• Integrate RAG into the Kaggle scenarios. (#262) (be0e48a)
• Kaggle loop update (Feature & Model) (#241) (4cf22a6)
• kaggle templates related (#287) (785fdc1)
• Model context for tuning and selection (#284) (f2831e7)
• Modify FactorRowCountEvaluator and FactorIndexEvaluator to return the ratio (#328) (8f43f8e)
• New competition - Optiver (#356) (3705efe)
• random forest for s3e11 (#347) (b57846d)
• refine the code in model description and fix some bugs in feedback.py (#288) (5b124d7)
• refine the template in several Kaggle competitions (#343) (034f238)
• Revise to support better hypothesis proposal (#390) (c55ec0a)
• show workspace in demo (#348) (ddf567c)
• support Multi output (#330) (3d36c45)
• Supporting COVID-19 competition (#374) (a1b63db)
• supporting Mnist competition (#375) (e958a34)
• Supporting Model Specifications (#319) (e126471)
• supporting various Kaggle competitions & scenarios for RD-Agent (#409) (75eea22)
• template for kaggle (#308) (ff97cf0)
• use auto gen seed when using LLM cache (#441) (ca15365)
• use unified pickle cacher & move llm config into a isolated config (#424) (2879ecf)
• xgboost gpu accelerate (#359) (56a5b8f)

Bug Fixes

• a bug of developer& edit s4e8 template (#338) (f12ce72)
• actively raised errors aer also considered as negative feedback. (#268) (46ec908)
• bug of saving preprocess cache files (#310) (5fb0608)
• cache (#383) (f2a6e75)
• change css tag of kaggle competition info crawler (#306) (1e3d38b)
• debug dsagent (#387) (8fe9511)
• eval_method cannot catch run factor error (#260) (2aaab31)
• fix a bug in competition metric evaluation (#407) (94c47d6)
• fix a bug in mini case (#389) (e75bb57)
• fix a bug in model tuning feedback (#316) (8aa088d)
• fix a bug in scenario.py (#388) (999a1eb)
• fix a bug in the format of the model input (#327) (8f0574e)
• fix a small bug in cache using module name and function name as unique folder name (#429) (4f8134a)
• fix a typo (#362) (9fafabd)
• fix cache result logic (#430) (5e34263)
• fix command injection (#421) (52f30a6)
• fix json load error (#386) (bba55fb)
• fix some bugs in feedback.py and refine the prompt (#292) (d834052)
• fix some bugs in knowledge base (#378) (fa6ff8e)
• fix some bugs in rag (#399) (194215c)
• fix some bugs in the entire loop (#274) (8a564ec)
• fix some errors in scenario.py, proposal.py and runner.py and several complex competition scenarios(#365) (2e383b1)
• improve_execution_time_in_kaggle_loop (#279) (4c8f998)
• kaggle data mount problem (#297) (795df31)
• Optiver fixes (#357) (b054017)
• partial bug in bench (#368) (af9808f)
• preprocess output format & some mistake in spelling (#358) (b8b2cd6)
• rag save file (#385) (1cb01dd)
• raise error in demo when no Metric in a Loop (#313) (e46a78e)
• refactor Bench (#302) (78a87f6)
• refine some codes (#353) (866c2e6)
• refine the prompt (#286) (77966c4)
• refine the ucb algorithm (#406) (14f7d97)
• revert model and make SOTA model available to COSTEER (#351) (3b7437b)
• stop using markup in docker env print (#336) (3009889)
• support seed and fix absolute path (#278) (26352e1)
• template for kaggle foreset & s4e9 (#334) (2393a41)
• test kaggle method (#296) (91a6196)
• update code to fix a small bug in model cache md5 hash (#303) (b00e4dc)
• update new feature engineering code format (#272) (7850b80)
• Update prompts.yaml to constrain only one model type (#341) (5b5dfee)
• Update runner.py to fix a small bug (#282) (8aef3ab)
• Use fixed file name in model costeer & fixing cache (#311) (1f910a5)

Performance Improvements

• some small upgrade to factor costeer to improve the performance (#420) (9eb931f)

Reverts

• Revert feat: Factor Implement Search Enhancement (#294) (#305) (f663cf4)

0.2.1 (2024-09-10)

Bug Fixes

• default model value in config (#256) (c097585)
• fix_dotenv_error (#257) (923063c)
• readme (#248) (8cede22)

0.2.0 (2024-09-07)

Features

• add collect info (#233) (89f4af9)
• add cross validation for kaggle scenario (#236) (e0b03ba)
• add progress status for docker env (#215) (538d4ef)
• Added loop code for Kaggle scene. (#211) (975c327)
• Demo display effect and usage (#162) (8cf122a)
• piloting of the framework (#227) (e9b103e)
• support more models for kaggle scenario (#223) (e3a9659)
• update model_experiment.py to support basic EDA (#220) (bf2684c)

Bug Fixes

• fix some bugs in llm calling (#217) (7b010f8)
• package dependency. (#234) (46be295)
• remove useless line (#177) (64e9a8e)

0.1.0 (2024-08-09)

Features

• add entry for rdagent. (#187) (121b6d9)
• change ui entry (#197) (fa5d335)
• remove pdfs and enable online pdf readings (#183) (18c0501)

Bug Fixes

• Fix a fail href in readme (#189) (1b89218)
• fix quick start problem (#191) (44f61bf)
• update command line in readme.md (#192) (9c45d24)

0.0.1 (2024-08-08)

Features

• Add description for scenario experiments. (#174) (fbd8c6d)
• Added QlibFactorFromReportScenario and improved the report-factor loop. (#161) (882c79b)
• filter feature which is high correlation to former implemented features (#145) (e818326)
• Remove redundant 'key steps' section in frontend scene display. (#169) (e767005)
• streamlit webapp demo for different scenarios (#135) (d8da7db)
• Uploaded Documentation, Updated Prompts & Some Code for model demo (#144) (529f935)

Bug Fixes

• Add framework handling for task coding failure. (#176) (5e14fa5)
• Comprehensive update to factor extraction. (#143) (b5ea040)
• first round app folder cleaning (#166) (6a5a750)
• fix pickle problem (#140) (7ee4258)
• fix release CI (#165) (85d6a5e)
• fix release CI error (#160) (1c9f8ef)
• fix several bugs in data mining scenario (#147) (b233380)
• fix some small bugs in report-factor loop (#152) (a79f9f9)
• fix_release_ci_error (#150) (4f82e99)
• Fixed some bugs introduced during refactoring. (#167) (f8f1445)
• optimize some prompts in factor loop. (#158) (c2c1330)

Miscellaneous Chores

• release 0.0.1 (1feacd3)