Repository navigation
Several Questions for TinyMyo pre-training #10
Description
Thank you for your excellent work and code. I am interested in using your pertained models for research purposes.
But while I was doing pre-training with my EMG data, I got some questions compared to papers: https://arxiv.org/pdf/2512.15729
-
Question about normalization
The paper describes how input normalization is handled during pretraining, but when I checked the public code, some parts seemed slightly different from how I understood it.
Could you please clarify what the exact normalization method used in the paper was?
In particular, I would like to confirm whether Min-Max normalization was actually used, or whether there were any additional preprocessing or normalization steps that I may have missed. -
Question about RobustQuartile normalization
I also noticed that the codebase contains a RobustQuartile-type normalization.
Could you please clarify whether this was actually used for the TinyMyo pretraining experiments, or whether it was included only for other experiments, an earlier version, or as an alternative option?
In other words, I would like to know which normalization method should be considered the correct reference when reproducing the results reported in the paper. -
Question about the loss function
From the paper, I understood that both masked and unmasked reconstruction losses are used. However, in the public code, the loss used for optimization and the loss shown in training/validation logging seem potentially different, so I want to make sure I am interpreting this correctly.
When the paper reports the pretraining loss, does it refer to:
the masked loss only, or
the total loss consisting of masked loss plus weighted unmasked loss?
I would appreciate clarification on which definition is correct.
- Question about faithful reproduction of the paper
If I want to reproduce the paper as faithfully as possible using the public GitHub code, are there any parts among the following that must be modified separately to match the exact paper setting?
-normalization
-masking strategy
-reconstruction loss
-logging or model selection criterion
It is also possible that I may have misunderstood parts of the paper, so if possible, I would greatly appreciate it if you could briefly summarize the final pretraining setting actually used in the paper experiments.
Activity
Thoriri commentedon Apr 12, 2026
Hi SanghoYum.
Firstly, thank you so much for your interest in our work!
I have been talking about this with my colleagues, and we are working to answer you to the best of our ability (will come later this week). Just so you know, BioFoundation also includes works for other Foundation Models (such as FEMBA and LUNA). So some of the normalization you see in the repo is used by those and not TinyMyo. But this is also a bit on us not having the README clearer on this. I will rewrite the README later to make it clearer.
Answers to the other parts will come a bit later in the week!
MatteoFasulo commentedon Apr 22, 2026
Hi @SanghoYum .
Thank you for your questions.
I will focus on the points related to TinyMyo, as some aspects involve other models within BioFoundation.
First, I apologize for the confusion around certain TinyMyo configurations. This is not the original codebase in which it was developed, and during the adaptation process, some inconsistencies may have been introduced.
Please also note that we are currently in the review phase and are actively refining parts of the code to ensure full reproducibility and ease of experimentation. You can track progress in PR #8 .
input_normalization.normalize: True), and the task uses MinMaxNormalization.masked_loss + 0.1 x unmasked_loss.
You are also correct regarding logging: the current
val_lossreflects only the masked component. We plan to update this to log both individual and combined losses (see PR TinyMyo: Update hand gesture classification experiment details #8).val_loss(masked loss only) - will be fixed after closing TinyMyo: Update hand gesture classification experiment details #8 .We will soon release an updated version of TinyMyo with all configurations properly aligned. Reproducing our setup will then require only installing the dependencies and running a single command.
SanghoYum commentedon Apr 23, 2026
@MatteoFasulo Thank you for your answer! I still have reconstruction issue. I am basically pre-training 12 different EMG datasets, I padded channels to 16channels also. I have held experiments in many ways fixing normalization, adding attention mask and channel valid mask, etc. But the model couldn't still reconstruct the masked part :( Now I am trying to change Decoder layer it seems little bit light for 12 datasets.
Thoriri commentedon Oct 12, 2026
@SanghoYum I assume this was resolved in the reconstruction comment part, and therefore i'll close this there.