Code

Iterable dataset#42

Code

Open

lmeribal wants to merge 82 commits intomainturbo-llm/turbo-alignment:mainfrom iterable-datasetturbo-llm/turbo-alignment:iterable-datasetCopy head branch name to clipboard

+6,226-5,174

Commits

Commits on Sep 10, 2024

• Batched processing without .cat

  

  lmeribal

  committed
  d9058a3

• Accelerate device

  

  lmeribal

  committed
  313d06d

• batching in preprocessing

  

  lmeribal

  committed
  103777d

• fake commit

  

  lmeribal

  committed
  dd02fe6

• fixed error with device

  

  lmeribal

  committed
  3c718c1

• removed threadpool

  

  lmeribal

  committed
  aee3af5

• no gradient calc for clip inference

  

  lmeribal

  committed
  2544a00

• in progress

  

  lmeribal

  committed
  e13ddb5

• in progress

  

  lmeribal

  committed
  8c0f6cc

• accelerate processing

  

  lmeribal

  committed
  019591b

Commits on Sep 11, 2024

• saving single files

  

  lmeribal

  committed
  43d6ba6

• going back to the pt files

  

  lmeribal

  committed
  0e51337

• accelerator in dataset preprocessing

  

  lmeribal

  committed
  40241cc

• waiting for every process

  

  lmeribal

  committed
  70bfd2e

• context manager

  

  lmeribal

  committed
  a2e21d6

• accelerate parallel processing

  

  lmeribal

  committed
  b12af8c

• accelerate parallel processing

  

  lmeribal

  committed
  a690bc1

• debugging processer

  lmeribal

  committed
  fa40061

• in progress

  lmeribal

  committed
  f0c31f6

• batch processing

  lmeribal

  committed
  8a916a6

• in progress

  lmeribal

  committed
  e6e3047

• new method for file reader

  

  lmeribal

  committed
  39bc147

• in progress

  lmeribal

  committed
  79af8d9

Commits on Sep 12, 2024

• IterableDataset for multimodal pipeline

  

  lmeribal

  committed
  b63c973

• prettifier

  

  lmeribal

  committed
  90b736f

Commits on Sep 13, 2024

• iterable dataset

  

  lmeribal

  committed
  8b44d5f

Commits on Sep 15, 2024

• in progress

  lmeribal

  committed
  e4d0c9f

Commits on Sep 17, 2024

• in progress

  

  lmeribal

  committed
  6b7ae86

Commits on Sep 26, 2024

• attention pooling adapter

  

  lmeribal

  committed
  5f2a1ae

• in p

  

  Elisei Rykov

  committed
  cc3a444

Commits on Sep 27, 2024

• attention pooling with top k selection

  

  lmeribal

  committed
  18b4ca8

• adapter with top patches selector

  

  lmeribal

  committed
  b91da3e

Commits on Sep 28, 2024

• in progress

  

  Elisei Rykov

  committed
  15d013a

Commits on Oct 15, 2024

• Batched processing without .cat

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  b874062

• Accelerate device

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  83385d2

• batching in preprocessing

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  543b759

• fake commit

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  a3338d3

• fixed error with device

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  8206a02

• removed threadpool

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  be42bfd

• no gradient calc for clip inference

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  480d820

• in progress

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  73442c9

• in progress

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  ed034b5

• accelerate processing

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  0fc28b1

• saving single files

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  749112a

• going back to the pt files

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  40278d5

• accelerator in dataset preprocessing

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  a3c3974

• waiting for every process

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  22fcd10

• context manager

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  3885a5c

• accelerate parallel processing

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  8a042f3

• accelerate parallel processing

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  232c26e

• debugging processer

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  f66c836

• in progress

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  e54c227

• batch processing

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  0eb2a78

• in progress

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  7c21c47

• new method for file reader

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  732abaf

• in progress

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  5777ddb

• IterableDataset for multimodal pipeline

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  9ba75a1

• prettifier

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  41ec908

• iterable dataset

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  608d618

• in progress

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  1c4d8d7

• in progress

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  0c66ad0

• attention pooling adapter

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  a60c5bb

• in p

  

  Elisei Rykov

  committed
  5a8f575

• attention pooling with top k selection

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  e52c899

• adapter with top patches selector

  

  lmeribal

  authored and

  Elisei Rykov

  committed
  206747e

• in progress

  

  Elisei Rykov

  committed
  3833e52

• in progress

  

  Elisei Rykov

  committed
  1167a34

• merging branches

  

  lmeribal

  committed
  cc8742a

• fixed merging

  

  lmeribal

  committed
  77aeb56

• in progress

  

  lmeribal

  committed
  05a6db0

Commits on Oct 18, 2024

• in progress

  

  Elisei Rykov

  committed
  d5bee2d

• inference dataset fix

  

  lmeribal

  committed
  bdc5a5a

• in progress

  

  Elisei Rykov

  committed
  cf0479f

• in progress

  

  lmeribal

  committed
  edf9b0c

• upd pyproject

  

  lmeribal

  committed
  aa86002

Commits on Oct 19, 2024

• upd lock

  

  lmeribal

  committed
  cc53839

• in progress

  

  Elisei Rykov

  committed
  e81c667

• in progress

  

  Elisei Rykov

  committed
  af99b82

• in progress

  

  lmeribal

  committed
  7f184cc

• in progress

  

  Elisei Rykov

  committed
  fedb076

Commits on Oct 30, 2024

• in progress

  

  lmeribal

  committed
  5af2cbb

Commits on Nov 26, 2024

• in progress

  

  Elisei Rykov

  committed
  edb4403