Discussions

• 1 You must be logged in to vote

  💬

  Generate embeddings functionality?

  bawiz started Aug 9, 2024 in General

  

  

  

  

  2
• 3 You must be logged in to vote

  🙏

  [Q] Limited context size

  mcm007 asked Jul 24, 2024 in Q&A · Unanswered

  

  

  

  

  11
• 1 You must be logged in to vote

  🙏

  Load lora weights and TP

  bks5881 asked Aug 28, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Shared memory usage

  dancixx asked Aug 24, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Idefics2 inference

  dancixx asked Aug 20, 2024 in Q&A · Answered

  

  

  7
• 3 You must be logged in to vote

  📣

  Faster than llama.cpp on CUDA

  EricLBuehler announced Jul 23, 2024 in Announcements

  

  

  1
• 3 You must be logged in to vote

  💡

  Advanced inference engine features

  kir-gadjello started Apr 29, 2024 in Ideas

  

  

  

  

  18
• 3 You must be logged in to vote

  💡

  Idea: Support distributed computing inference serialized over layer-subset groups.

  ghchris2021 started Jul 3, 2024 in Ideas

  

  

  

  

  4
• 2 You must be logged in to vote

  🙏

  plan for GRPC ?

  Ranganaths asked May 15, 2024 in Q&A · Unanswered

  

  

  

  

  4
• 6 You must be logged in to vote

  📣

  Optimization plans

  EricLBuehler announced Mar 30, 2024 in Announcements

  

  

  

  

  10
• 2 You must be logged in to vote

  🙌

  My general advice / tips for compiling mistral.rs in WSL2

  joshuasundance-swca started Jul 8, 2024 in Show and tell

  

  

  0
• 2 You must be logged in to vote

  💡

  Idea: Support Vulkan / SYCL / OpenCL for inference improving coverage for other GPUs e.g. intel, amd.

  ghchris2021 started Jul 3, 2024 in Ideas

  

  

  0
• 1 You must be logged in to vote

  🙏

  models being mut (forward)

  bug Something isn't working

  jondot asked Jun 24, 2024 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  What are the differences between the llama block in this repo and the implementation in candle-transformer?

  chenwanqq asked Jun 23, 2024 in Q&A · Unanswered

  

  

  10
• 3 You must be logged in to vote

  💡

  Support ROCm

  nonetrix started May 16, 2024 in Ideas

  

  

  2
• 1 You must be logged in to vote

  🙏

  #238 I wanted to ask what software is @lucasavila00 using in this image to run mistral and see the layers inside of it ( looks like a gpu profiler)

  xXWarMachineRoXx asked Apr 29, 2024 in Q&A · Answered

  

  

  

  

  4