Qlib Highfreq Support & Highfreq DataHanlder/Operator/Processor Examples#222
Merged
you-n-g merged 18 commits intomicrosoft:mainfrom Jan 28, 2021
Merged
Qlib Highfreq Support & Highfreq DataHanlder/Operator/Processor Examples#222you-n-g merged 18 commits intomicrosoft:mainfrom
you-n-g merged 18 commits intomicrosoft:mainfrom
Conversation
bxdd
commented
Jan 27, 2021
bxdd
commented
Jan 27, 2021
qlib/data/data.py
Outdated
| H["c"][flag] = _calendar, _calendar_index | ||
| return _calendar, _calendar_index | ||
|
|
||
| def get_calendar_day(self, freq="day", future=False): |
Collaborator
Author
There was a problem hiding this comment.
get_calendar_day just for high-freq, if there are other ways to implement it
Collaborator
There was a problem hiding this comment.
We don't need to put this in qlib/data/data.py
you-n-g
reviewed
Jan 27, 2021
qlib/data/data.py
Outdated
| H["c"][flag] = _calendar, _calendar_index | ||
| return _calendar, _calendar_index | ||
|
|
||
| def get_calendar_day(self, freq="day", future=False): |
Collaborator
There was a problem hiding this comment.
We don't need to put this in qlib/data/data.py
you-n-g
reviewed
Jan 27, 2021
examples/high_freq/workflow.py
Outdated
| # use default data | ||
| provider_uri = "/nfs_data/qlib_data/yahoo_high_qlib" # target_dir | ||
| qlib.init( | ||
| provider_uri=provider_uri, |
examples/high_freq/workflow.py
Outdated
| qlib.init( | ||
| provider_uri=provider_uri, | ||
| custom_ops=[DayFirst, DayLast, FFillNan, Date, Select, IsNull], | ||
| redis_port=-1, |
Collaborator
There was a problem hiding this comment.
Default in most case
- Enable expression cache , disable dataset cache.
examples/high_freq/workflow.py
Outdated
| qlib.init( | ||
| provider_uri=provider_uri, | ||
| custom_ops=[DayFirst, DayLast, FFillNan, Date, Select, IsNull], | ||
| redis_port=-1, |
Collaborator
There was a problem hiding this comment.
disable expression cache in this case
examples/high_freq/workflow.py
Outdated
| # use default data | ||
| provider_uri = "/nfs_data/qlib_data/yahoo_high_qlib" # target_dir | ||
| qlib.init( | ||
| provider_uri=provider_uri, |
Collaborator
There was a problem hiding this comment.
SPEC_CONF
HIGH_FREQ_CONFIG
``qlib.init(**HIGH_FREQ_CONFIG, **SPEC_CONF)`
examples/high_freq/workflow.py
Outdated
| # use default data | ||
| provider_uri = "/nfs_data/qlib_data/yahoo_high_qlib" # target_dir | ||
| qlib.init( | ||
| provider_uri=provider_uri, |
Collaborator
There was a problem hiding this comment.
provider_uri in HIGH_FREQ_CONFIG
| if DROP_LOAD_DATASET: | ||
|
|
||
| ##=============dump dataset============= | ||
| dataset.to_pickle(path="dataset.pkl") |
Collaborator
There was a problem hiding this comment.
import fire
class HFWorkflow:
def XXXX
def case1:
def case2:
you-n-g
reviewed
Jan 28, 2021
| fields += [ | ||
| "{0}/Ref(DayLast({1}), 240)".format( | ||
|
|
||
| def get_04_price_feature(price_field): |
Collaborator
There was a problem hiding this comment.
get_04_price_feature & get_59_price_feature are really weird names.
examples/highfreq/highfreq_ops.py
Outdated
| class FFillNan(ElemOperator): | ||
| def __init__(self, feature): | ||
| super(FFillNan, self).__init__(feature, "fill_nan") | ||
| super(FFillNan, self).__init__(feature) |
Collaborator
There was a problem hiding this comment.
These init function are no longer required.
examples/highfreq/workflow.py
Outdated
| provider_uri = QLIB_INIT_CONFIG.get("provider_uri") | ||
| if not exists_qlib_data(provider_uri): | ||
| print(f"Qlib data is not found in {provider_uri}") | ||
| sys.path.append(str(Path(__file__).resolve().parent.parent.parent.joinpath("scripts"))) |
Collaborator
There was a problem hiding this comment.
from qlib.tests.data import GetData
you-n-g
reviewed
Jan 28, 2021
examples/workflow_by_code.py
Outdated
| GetData().qlib_data(target_dir=provider_uri, region=REG_CN) | ||
|
|
||
| qlib.init(provider_uri=provider_uri, region=REG_CN) | ||
| qlib.init(provider_uri=provider_uri, region=REG_CN, redis_port=-1) |
gaogen123
pushed a commit
to gaogen123/qlib
that referenced
this pull request
Sep 13, 2025
Qlib Highfreq Support & Highfreq DataHanlder/Operator/Processor Examples
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Qlib Highfreq Support & Highfreq DataHanlder/Operator/Processor Examples
Motivation and Context
How Has This Been Tested?
pytest qlib/tests/test_all_pipeline.pyunder upper directory ofqlib.Screenshots of Test Results (if appropriate):
Types of changes