Skip to content

Commit

Permalink
Fix a device issue.
Browse files Browse the repository at this point in the history
  • Loading branch information
Negar Foroutan Eghlidi committed Aug 22, 2024
1 parent 8b68126 commit 42faa37
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/nanotron/trainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -691,7 +691,7 @@ def validation_step(self, dataloader: Iterator[Dict[str, Union[torch.Tensor, Ten
if not lang_losses[
lang
]: # If the list is empty --> Set local language loss to -1 to exclude it from the global computation
lang_losses[lang] = torch.tensor(-1, dtype=torch.float32)
lang_losses[lang] = torch.tensor(-1, dtype=torch.float32, device="cuda")
else: # If we have at least 1 loss from a given language --> compute local language loss mean
lang_losses[lang] = torch.mean(torch.stack(lang_losses[lang]))

Expand Down

0 comments on commit 42faa37

Please sign in to comment.