Skip to content

API ergonomics to allow to combine your features with daily price data #20

@jaapterwoerds

Description

@jaapterwoerds

When playing with the opensignals for a while I noticed I often want to merge / join my own data and calculate trends on combination of columns which seems pretty hard with the current API which return the weekly targets merged with the features. I've split the get_data method so I can do my own feature engineering based on other data but possibly making use of the prices features from opensignals, something like this:

def get_data(path, provider):    
    db_dir = Path(path)
    logging.basicConfig(level=logging.INFO)
    provider.download_data(db_dir)
    features_generators = [
        RSI(num_days=20, interval=7, variable='adj_close'),
        RSI(num_days=14, interval=14, variable='adj_close'),
        RSI(num_days=7, interval=20, variable='adj_close'),
        SMA(num_days=20, interval=7, variable='adj_close'),
        SMA(num_days=14, interval=14, variable='adj_close'),
        SMA(num_days=7, interval=21, variable='adj_close'),
        RSI(num_days=1, interval=5, variable='volume'),
        RSI(num_days=2, interval=5, variable='volume'),
        RSI(num_days=3, interval=5, variable='volume'),
        SMA(num_days=1, interval=5, variable='volume'),
        SMA(num_days=2, interval=5, variable='volume'),
        SMA(num_days=7, interval=20, variable='volume'),        
    ]
    ticker_data  = common.get_ticker_data(db_dir)

    feature_names = []
    for features_generator in features_generators:
        ticker_data, feature_names_aux = features_generator.generate_features(ticker_data)
        feature_names.extend(feature_names_aux)
    
    # Custom feature engineering with own features making use of the generated price features
    features, feature_names = feature_engineering(ticker_data, tickers)   
    
    # Merge with target to get weekly data
    return process_ml_data(features, feature_names, features_generators=features_generators)

def process_ml_data(
        ticker_data,
        feature_names,
        features_generators=None,
        last_friday=datetime.today() - relativedelta(weekday=FR(-1)),
        target='target_20d'):
    """merge our feature data with Numerai targets"""
   ...

I was wondering if the scope of this library should allow something like this. I would love to hear other viewpoints on this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions