Forecasting QA: Question-Answering Style Value Prediction¶

This notebook demonstrates the Forecasting QA task mode in TwinWeaver.

While the default forward_conversion mode ("forecasting") asks the model to predict exact future values, the Forecasting QA mode bins continuous target values into discrete categories (e.g., A, B, C) and asks the model to predict the correct bin — turning regression into classification.

This can be useful when:

• Exact numeric predictions are noisy or unreliable
• You want the model to reason about value ranges instead of point estimates
• You want to combine both forecasting and QA tasks (mode "both")

How it works¶

The ConverterForecastingQA sub-converter:

1. Computes bin edges from the variable statistics (setup_statistics() on the forecasting splitter)
2. Maps each target value to a lettered bin (A, B, C, …)
3. Generates a prompt listing the bin definitions and asks the model to choose the right bin
4. The target answer uses the bin letter instead of the raw numeric value

Selecting the mode¶

The forward_conversion method accepts an override_mode_to_select_forecasting parameter:

• "forecasting" (default): numeric value prediction
• "forecasting_qa": bin-based QA prediction
• "both": includes both a forecasting and a forecasting QA task
• None: randomly selects one of the above at each call

Setup¶

Load libraries and example data.

In [ ]:

import pandas as pd
import numpy as np

from twinweaver import (
    DataSplitterForecasting,
    DataSplitterEvents,
    DataSplitter,
    DataManager,
    ConverterInstruction,
    Config,
)

import pandas as pd import numpy as np from twinweaver import ( DataSplitterForecasting, DataSplitterEvents, DataSplitter, DataManager, ConverterInstruction, Config, )

In [ ]:

df_events = pd.read_csv("../../example_data/events.csv")
df_constant = pd.read_csv("../../example_data/constant.csv")
df_constant_description = pd.read_csv("../../example_data/constant_description.csv")

df_events = pd.read_csv("../../example_data/events.csv") df_constant = pd.read_csv("../../example_data/constant.csv") df_constant_description = pd.read_csv("../../example_data/constant_description.csv")

Configuration¶

Set up the config, data manager, splitters, and converter.

Important: variable_stats from DataSplitterForecasting.setup_statistics() must be passed to ConverterInstruction for the QA mode to work — it provides the bin edges.

In [ ]:

config = Config()

config.split_event_category = "lot"
config.event_category_forecast = ["lab"]
config.event_category_events_prediction_with_naming = {
    "death": "death",
    "progression": "next progression",
}
config.constant_columns_to_use = ["birthyear", "gender", "histology", "smoking_history"]
config.constant_birthdate_column = "birthyear"

config = Config() config.split_event_category = "lot" config.event_category_forecast = ["lab"] config.event_category_events_prediction_with_naming = { "death": "death", "progression": "next progression", } config.constant_columns_to_use = ["birthyear", "gender", "histology", "smoking_history"] config.constant_birthdate_column = "birthyear"

In [ ]:

dm = DataManager(config=config)
dm.load_indication_data(
    df_events=df_events,
    df_constant=df_constant,
    df_constant_description=df_constant_description,
)
dm.process_indication_data()
dm.setup_unique_mapping_of_events()
dm.setup_hold_out_sets(validation_split=0.1, test_split=0.1)
dm.infer_var_types()

data_splitter_events = DataSplitterEvents(
    dm,
    config=config,
    max_length_to_sample=pd.Timedelta(weeks=104),
    min_length_to_sample=pd.Timedelta(weeks=1),
)
data_splitter_events.setup_variables()

data_splitter_forecasting = DataSplitterForecasting(
    data_manager=dm,
    config=config,
    max_forecasted_trajectory_length=pd.Timedelta(days=90),
)
# setup_statistics() computes per-variable quantile bin edges used by the QA mode
data_splitter_forecasting.setup_statistics()

data_splitter = DataSplitter(
    data_splitter_events=data_splitter_events,
    data_splitter_forecasting=data_splitter_forecasting,
)

converter = ConverterInstruction(
    nr_tokens_budget_total=8192,
    config=config,
    dm=dm,
    variable_stats=data_splitter_forecasting.variable_stats,  # Required for QA mode
)

dm = DataManager(config=config) dm.load_indication_data( df_events=df_events, df_constant=df_constant, df_constant_description=df_constant_description, ) dm.process_indication_data() dm.setup_unique_mapping_of_events() dm.setup_hold_out_sets(validation_split=0.1, test_split=0.1) dm.infer_var_types() data_splitter_events = DataSplitterEvents( dm, config=config, max_length_to_sample=pd.Timedelta(weeks=104), min_length_to_sample=pd.Timedelta(weeks=1), ) data_splitter_events.setup_variables() data_splitter_forecasting = DataSplitterForecasting( data_manager=dm, config=config, max_forecasted_trajectory_length=pd.Timedelta(days=90), ) # setup_statistics() computes per-variable quantile bin edges used by the QA mode data_splitter_forecasting.setup_statistics() data_splitter = DataSplitter( data_splitter_events=data_splitter_events, data_splitter_forecasting=data_splitter_forecasting, ) converter = ConverterInstruction( nr_tokens_budget_total=8192, config=config, dm=dm, variable_stats=data_splitter_forecasting.variable_stats, # Required for QA mode )

Generate Splits¶

Get training splits for a patient.

In [ ]:

np.random.seed(42)

patientid = dm.all_patientids[2]
patient_data = dm.get_patient_data(patientid)

forecasting_splits, events_splits, reference_dates = data_splitter.get_splits_from_patient_with_target(
    patient_data,
    forecasting_nr_samples_per_split=4,
    forecasting_filter_outliers=False,
    max_num_splits_per_split_event=2,
    events_max_nr_samples_per_split=3,
)

np.random.seed(42) patientid = dm.all_patientids[2] patient_data = dm.get_patient_data(patientid) forecasting_splits, events_splits, reference_dates = data_splitter.get_splits_from_patient_with_target( patient_data, forecasting_nr_samples_per_split=4, forecasting_filter_outliers=False, max_num_splits_per_split_event=2, events_max_nr_samples_per_split=3, )

Mode 1: Default Forecasting (numeric predictions)¶

By default, forward_conversion uses override_mode_to_select_forecasting="forecasting", which generates tasks that ask the model to predict exact numeric values.

In [ ]:

np.random.seed(42)
split_idx = 0

p_forecasting = converter.forward_conversion(
    forecasting_splits=forecasting_splits[split_idx],
    event_splits=events_splits[split_idx],
    # override_mode_to_select_forecasting="forecasting"  # this is the default
)

print("=" * 80)
print("INSTRUCTION (last 1500 chars):")
print("=" * 80)
print(p_forecasting["instruction"][-1500:])
print()
print("=" * 80)
print("ANSWER:")
print("=" * 80)
print(p_forecasting["answer"])

np.random.seed(42) split_idx = 0 p_forecasting = converter.forward_conversion( forecasting_splits=forecasting_splits[split_idx], event_splits=events_splits[split_idx], # override_mode_to_select_forecasting="forecasting" # this is the default ) print("=" * 80) print("INSTRUCTION (last 1500 chars):") print("=" * 80) print(p_forecasting["instruction"][-1500:]) print() print("=" * 80) print("ANSWER:") print("=" * 80) print(p_forecasting["answer"])

Mode 2: Forecasting QA (bin-based predictions)¶

Setting override_mode_to_select_forecasting="forecasting_qa" activates the QA mode. The prompt now lists bin definitions (e.g., a = Bin (-inf, 5.2]) and asks the model to output the bin letter instead of a raw number.

In [ ]:

np.random.seed(42)
split_idx = 0

p_qa = converter.forward_conversion(
    forecasting_splits=forecasting_splits[split_idx],
    event_splits=events_splits[split_idx],
    override_mode_to_select_forecasting="forecasting_qa",
)

print("=" * 80)
print("INSTRUCTION (last 2000 chars):")
print("=" * 80)
print(p_qa["instruction"][-2000:])
print()
print("=" * 80)
print("ANSWER:")
print("=" * 80)
print(p_qa["answer"])

np.random.seed(42) split_idx = 0 p_qa = converter.forward_conversion( forecasting_splits=forecasting_splits[split_idx], event_splits=events_splits[split_idx], override_mode_to_select_forecasting="forecasting_qa", ) print("=" * 80) print("INSTRUCTION (last 2000 chars):") print("=" * 80) print(p_qa["instruction"][-2000:]) print() print("=" * 80) print("ANSWER:") print("=" * 80) print(p_qa["answer"])

Inspect the bin definitions¶

The metadata returned by the conversion includes the bin mapping for each variable. These are derived from quantiles computed by setup_statistics().

In [ ]:

# The target meta contains the category splits for each variable
for task_meta in p_qa["meta"]["target_meta_detailed"]:
    if "category_splits" in task_meta:
        print("Variable bin definitions:")
        for var, bins in task_meta["category_splits"].items():
            print(f"  {var}:")
            for letter, bin_range in bins.items():
                print(f"    {letter} = {bin_range}")

# The target meta contains the category splits for each variable for task_meta in p_qa["meta"]["target_meta_detailed"]: if "category_splits" in task_meta: print("Variable bin definitions:") for var, bins in task_meta["category_splits"].items(): print(f" {var}:") for letter, bin_range in bins.items(): print(f" {letter} = {bin_range}")

Mode 3: Both (forecasting + QA in one prompt)¶

Setting override_mode_to_select_forecasting="both" includes both a numeric forecasting task and a QA-style bin prediction task in the same multi-task prompt. This encourages the model to learn complementary representations.

In [ ]:

np.random.seed(42)
split_idx = 0

p_both = converter.forward_conversion(
    forecasting_splits=forecasting_splits[split_idx],
    event_splits=events_splits[split_idx],
    override_mode_to_select_forecasting="both",
)

print("=" * 80)
print("INSTRUCTION (last 3000 chars):")
print("=" * 80)
print(p_both["instruction"][-3000:])
print()
print("=" * 80)
print("ANSWER:")
print("=" * 80)
print(p_both["answer"])

np.random.seed(42) split_idx = 0 p_both = converter.forward_conversion( forecasting_splits=forecasting_splits[split_idx], event_splits=events_splits[split_idx], override_mode_to_select_forecasting="both", ) print("=" * 80) print("INSTRUCTION (last 3000 chars):") print("=" * 80) print(p_both["instruction"][-3000:]) print() print("=" * 80) print("ANSWER:") print("=" * 80) print(p_both["answer"])

Compare the task types generated¶

Let's inspect which task types were included in each mode.

In [ ]:

for label, result in [("forecasting", p_forecasting), ("forecasting_qa", p_qa), ("both", p_both)]:
    task_types = [m["task_type"] for m in result["meta"]["target_meta_detailed"]]
    print(f"Mode '{label}': {task_types}")

for label, result in [("forecasting", p_forecasting), ("forecasting_qa", p_qa), ("both", p_both)]: task_types = [m["task_type"] for m in result["meta"]["target_meta_detailed"]] print(f"Mode '{label}': {task_types}")

Reverse Conversion¶

The reverse conversion works for all modes. It parses the model output back into structured data, regardless of whether the target was numeric or bin-based.

In [ ]:

date = reference_dates["date"][0]

# Reverse convert the QA mode output
return_list = converter.reverse_conversion(p_qa["answer"], dm, date)

return_list[2]["result"]

date = reference_dates["date"][0] # Reverse convert the QA mode output return_list = converter.reverse_conversion(p_qa["answer"], dm, date) return_list[2]["result"]

Variable Statistics¶

The bin edges come from the variable_stats DataFrame computed during setup_statistics(). You can inspect them directly:

In [ ]:

data_splitter_forecasting.variable_stats

data_splitter_forecasting.variable_stats