Resolved alibi bias issue due to porting flat PA pr

[tannervoas742]

Requires associated changes on vllm-fork PR

This adds back ALiBI biases to forward_decode for models which required ALiBI. This was broken in PR #169.

Below you can see the results on mpt-7B model which uses ALiBI. The first example is the current code on main. Second is the fixed one. And the third is when reverting the decode attention back to paged_attention_v1 before #169 which produces the best results in my opinion.

• main:
  • Prompt: 'Hello, my name is'
  • Generated text: ' Mr. Andy. My students know me as a calm and fun teacher who can make things collectively. They also say I think about the students with your minds. I am good brain enough.\nBest well, my students say takes are my teaching and are all exactly on me. Great. My favorite.\nMy timeHappy So I got It interested students’s believe talents. Because I good well. I am too always So, hate i.I’s space for I(y autumn I my guide be interesting.'
• this PR:
  • Prompt: 'Hello, my name is'
  • Generated text: " Mr. Andy. My students know me as a calm and fun teacher who can make things as lively as possible in a classroom. I'm enthusiastic about education and I am always eager to learn more about the cultures of other countries. I believe teaching and learning is like dancing, you can never be better than the last time you were on your own feet and there are steps we can all follow. I look forward to seeing you at IIC!"
• reverted attention:
  • Prompt: 'Hello, my name is'
  • Generated text: ' Mr. Neil Banfield. Are you looking to begin the startup of your business? You’ve come to the right place, because TrustLaw And Co. is here to help you. We offer the best range of legal takes, including general business, contracts and dispute resolution. We have extensive knowledge when it comes to the above-mentioned areas. Here is why you should consider hiring us.\nLegal services we combine all the aspects of legal services in so a client only has to one legal service during their matter. When'

[tannervoas742]

I provided two fixes for ALiBI. One is #438 paired with HabanaAI/vllm-hpu-extension#19 which adds back ALiBI biases to flat_pa. The other is #437 paired with HabanaAI/vllm-hpu-extension#18 that uses the old attention mechanism (paged_attention_v1) if ALiBI is required.