ValueError: df does not contain all expected columns. Missing columns: ['id'] when using predict_df

[Laxmi530]

I am trying to generate forecasts using ChronosPipeline.predict_df, but I am getting the following error:
ValueError: df does not contain all expected columns. Missing columns: ['id']

Below is the code i am trying

pipeline: Chronos2Pipeline = BaseChronosPipeline.from_pretrained(model_amazon_chronos_2, device_map="cpu")

feature_df = df_daily.drop(columns="adj_close")

pred_df = pipeline.predict_df(
    df_daily,
    future_df=feature_df,
    prediction_length=10,
    quantile_levels=[0.1, 0.5, 0.9],
    id_column="id",
    timestamp_column="datetime",
    target="adj_close",
)

My input dataframe (df_daily) contains the following columns:

datetime open high low close adj_close volume

There is no id column because this dataset represents a single stock data.

How should this be handled when working with a single stock data ?

Thanks for the great work on Chronos.
Please let me know if a minimal reproducible example or additional details would be helpful.

[abdulfatir]

id is a required column. If you have a single series, you can just add a dummy ID column.

[Laxmi530]

Thanks @abdulfatir for the clarification.

Just to confirm my understanding: for a single series, adding a dummy id column (for example, a constant value like stock_1) is sufficient to satisfy the API requirement.

Could you please clarify how the id column is actually used internally during prediction?
Specifically:

• Does the id play any role in the model’s learned representations or forecasting logic?

• Or is it only used as a grouping key to support multi-series forecasting, with no impact on predictions when there is only a single series?

Understanding this would help ensure that adding a dummy id does not unintentionally affect forecast behavior for single-stock use cases.

Thanks again for the guidance and for maintaining Chronos.

[abdulfatir]

It's not used the model itself. It's used by the API to return the forecasts correctly for multiple series.

[Laxmi530]

Thanks @abdulfatir for the clarification.

[Laxmi530]

Hi @abdulfatir ,
As per your suggestion i added a id column to the dataframe but now i am getting below error. Can you please guide me how can i fix the error. JKIL is the ticker of the company.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[16], [line 2](vscode-notebook-cell:?execution_count=16&line=2)
      1 # Generate predictions with covariates
----> [2](vscode-notebook-cell:?execution_count=16&line=2) pred_df = pipeline.predict_df(
      3     df_daily,
      4     future_df=feature_df,
      5     prediction_length=10,               # Number of steps to forecast
      6     quantile_levels=[0.1, 0.5, 0.9],    # Quantiles for probabilistic forecast
      7     id_column="id",                     # Column identifying different time series
      8     timestamp_column="datetime",        # Column with datetime information
      9     target="adj_close",                 # Column(s) with time series values to predict
     10 )

File ~/Coding Stuff/env/lib/python3.12/site-packages/chronos/base.py:196, in BaseChronosPipeline.predict_df(self, df, id_column, timestamp_column, target, prediction_length, quantile_levels, validate_inputs, **predict_kwargs)
    193 if prediction_length is None:
    194     prediction_length = self.model_prediction_length
--> [196](https://file+.vscode-resource.vscode-cdn.net/home/user/Downloads/Working/Stock%20Prediction/~/Coding%20Stuff/env/lib/python3.12/site-packages/chronos/base.py:196) inputs, original_order, prediction_timestamps = convert_df_input_to_list_of_dicts_input(
    197     df=df,
    198     future_df=None,
    199     id_column=id_column,
    200     timestamp_column=timestamp_column,
    201     target_columns=[target],
    202     prediction_length=prediction_length,
    203     validate_inputs=validate_inputs,
    204 )
    206 # NOTE: any covariates, if present, are ignored here
    207 context = [torch.tensor(item["target"]).squeeze(0) for item in inputs]  # squeeze the extra variate dim

File ~/Coding Stuff/env/lib/python3.12/site-packages/chronos/df_utils.py:246, in convert_df_input_to_list_of_dicts_input(df, future_df, target_columns, prediction_length, id_column, timestamp_column, validate_inputs)
    243 import pandas as pd
    245 if validate_inputs:
--> [246](https://file+.vscode-resource.vscode-cdn.net/home/user/Downloads/Working/Stock%20Prediction/~/Coding%20Stuff/env/lib/python3.12/site-packages/chronos/df_utils.py:246)     df, future_df, freq, series_lengths, original_order = validate_df_inputs(
    247         df,
    248         future_df=future_df,
    249         id_column=id_column,
    250         timestamp_column=timestamp_column,
    251         target_columns=target_columns,
    252         prediction_length=prediction_length,
    253     )
    254 else:
    255     # Get the original order of time series IDs
    256     original_order = df[id_column].unique()

File ~/Coding Stuff/env/lib/python3.12/site-packages/chronos/df_utils.py:168, in validate_df_inputs(df, future_df, target_columns, prediction_length, id_column, timestamp_column)
    166     timestamps = timestamp_index[start_idx : start_idx + length]
    167     series_id = df[id_column].iloc[start_idx]
--> [168](https://file+.vscode-resource.vscode-cdn.net/home/user/Downloads/Working/Stock%20Prediction/~/Coding%20Stuff/env/lib/python3.12/site-packages/chronos/df_utils.py:168)     all_freqs.append(validate_freq(timestamps, series_id))
    169     start_idx += length
    171 if len(set(all_freqs)) > 1:

File ~/Coding Stuff/env/lib/python3.12/site-packages/chronos/df_utils.py:153, in validate_df_inputs.<locals>.validate_freq(timestamps, series_id)
    151 freq = pd.infer_freq(timestamps)
    152 if not freq:
--> [153](https://file+.vscode-resource.vscode-cdn.net/home/user/Downloads/Working/Stock%20Prediction/~/Coding%20Stuff/env/lib/python3.12/site-packages/chronos/df_utils.py:153)     raise ValueError(f"Could not infer frequency for series {series_id}")
    154 return freq

ValueError: Could not infer frequency for series JKIL

Below is the code i am using

import pandas as pd
from chronos import Chronos2Pipeline, BaseChronosPipeline

pipeline: Chronos2Pipeline = BaseChronosPipeline.from_pretrained(model_amazon_chronos_2, device_map="cpu")
company_name = 'J.Kumar Infraprojects Limited'
df_daily = get_whole_stock_data(company_name, interval="1d")
df_daily['id'] = str(get_ticker(company_name))

feature_df = df_daily.drop(columns="adj_close")

pred_df = pipeline.predict_df(
    df_daily,
    future_df=feature_df,
    prediction_length=10,               # Number of steps to forecast
    quantile_levels=[0.1, 0.5, 0.9],    # Quantiles for probabilistic forecast
    id_column="id",                     # Column identifying different time series
    timestamp_column="datetime",        # Column with datetime information
    target="adj_close",                 # Column(s) with time series values to predict
)

Thanks again for the guidance and for maintaining Chronos.

[shchur]

Timestamps in the data must have a regular frequency. LLMs are really good at debugging issues like these and can generate code to fix the issue for you - you can just copy paste the entire stack trace of the error and usually get a solution in one go