Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About the input data #2

Open
mickey-cool opened this issue Dec 24, 2024 · 3 comments
Open

About the input data #2

mickey-cool opened this issue Dec 24, 2024 · 3 comments

Comments

@mickey-cool
Copy link

mickey-cool commented Dec 24, 2024

Based on your code, the dataset you used was a csv. However, I am still confused about how did you organize all the data in the csv, because It was a total failure when I tried so many times to use my csv dataset as input of the model with your code. Would you please offer me a sample of your data as an example?Thank you!

@shinfxh
Copy link
Collaborator

shinfxh commented Dec 24, 2024

Sorry for the confusion, maybe it helps to refer to this previous issue. When loaded, the dataframe should already be preprocessed into shape $N \times C$ where $N$ is the number of samples and $C$ is the context length.

@mickey-cool
Copy link
Author

Traceback (most recent call last):
File "/media/strawberry/codes/timesfm_fin/src/main.py", line 68, in
app.run(main)
File "/media/watermelon/miniconda3/envs/tsfm/lib/python3.10/site-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/media/watermelon/miniconda3/envs/tsfm/lib/python3.10/site-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "/media/strawberry/codes/timesfm_fin/src/main.py", line 63, in main
train.train_and_evaluate(tfm, config, workdir)
File "/media/strawberry/codes/timesfm_fin/src/train.py", line 466, in train_and_evaluate
replicated_jax_states, step_fun_out = p_train_step(
File "/media/strawberry/codes/timesfm_fin/src/train.py", line 111, in train_step
return trainer_lib.train_step_single_learner(
File "/media/watermelon/miniconda3/envs/tsfm/lib/python3.10/site-packages/paxml/trainer_lib.py", line 1093, in train_step_single_learner
var_weight_hparams = model.abstract_init_with_metadata(inputs)
File "/media/watermelon/miniconda3/envs/tsfm/lib/python3.10/site-packages/praxis/base_layer.py", line 2042, in abstract_init_with_metadata
variables_abstract = self._abstract_init(
File "/media/watermelon/miniconda3/envs/tsfm/lib/python3.10/site-packages/praxis/base_layer.py", line 2025, in _abstract_init
variables_abstract = jax.eval_shape(
File "/media/watermelon/miniconda3/envs/tsfm/lib/python3.10/site-packages/praxis/base_model.py", line 111, in call
predictions = self.compute_predictions(input_batch)
File "/media/strawberry/codes/timesfm_fin/src/train.py", line 312, in compute_predictions
return self.core_layer(new_input_batch)
File "/media/watermelon/miniconda3/envs/tsfm/lib/python3.10/site-packages/timesfm/patched_decoder.py", line 391, in call
model_input, patched_padding, stats, _ = self._preprocess_input(
File "/media/watermelon/miniconda3/envs/tsfm/lib/python3.10/site-packages/timesfm/patched_decoder.py", line 329, in _preprocess_input
patched_pads = es.jax_einshape(
File "/media/watermelon/miniconda3/envs/tsfm/lib/python3.10/site-packages/einshape/src/jax/jax_ops.py", line 63, in einshape
return _JaxBackend().exec(equation, value, value.shape, **index_sizes)
File "/media/watermelon/miniconda3/envs/tsfm/lib/python3.10/site-packages/einshape/src/backend.py", line 80, in exec
ops = engine.generate_ops(equation, shape, **index_sizes)
File "/media/watermelon/miniconda3/envs/tsfm/lib/python3.10/site-packages/einshape/src/engine.py", line 175, in generate_ops
raise ValueError(
ValueError: Dimension to ungroup is not divisible by its index sizes. Group "(np)" expects size 532, but its indices "p" have combined specified size 32.

I always get this result, please help me, thank you!

@shinfxh
Copy link
Collaborator

shinfxh commented Dec 28, 2024

Can you give an example of the csv file you used?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants