How you calculated attention mask?

[tumurzakov]

I'm reading attention.py, function promptsepcalc and cant understan where mask actually calculated.

How you extract this mask?

I'm try to use this technic with animatediff. Mask introduces occlusions. This not visible on single image but it visible on video. I want to calculate attention mask as you did, but cant understand how

[hako-mikan]

Calculation of attention maps is operated in main_forward in attention.py

    if userpp and step > 0:
        for b in range(attn.shape[0] // 8):
            if pmaskshw == []:
                pmaskshw = [(height,width)]
            elif (height,width) not in pmaskshw:
                pmaskshw.append((height,width))

            for t in tokens:
                power = 4 if isxl else 1.2
                add = attn[8*b:8*(b+1),:,t[0]:t[0]+len(t)]**power
                add = torch.sum(add,dim = 2)
                t = f"{t}-{b}"         
                if t not in pmasks:
                    pmasks[t] = add
                else:
                    if pmasks[t].shape[1] != add.shape[1]:
                        add = add.view(8,height,width)
                        add = F.resize(add,pmaskshw[0])
                        add = add.reshape_as(pmasks[t])

                    pmasks[t] = pmasks[t] + add

Here, consolidating the attn read from each layer into a single map. I pre-count the tokens of the word I want to inquire about, and extract only the attn corresponding to those word tokens. As it stands, the attn is unusable, so I'm implementing a certain degree of cutoff, etc.

I have been thinking that this technique might be usable in animation, and recently tried creating an animation utilizing this.
https://twitter.com/hakomikanx/status/1700522316730671263